Computer network intrusion detection systems (sensors) rely mainly on signature-based sensing for malicious traffic detection. At a basic level, a signature is a definitive sequence of network communication data-streams (bits) that are characteristic of a known attack pattern. Generally, every known exploit or internet attack will have many signatures written for signature-based sensors' use in intrusion detection. A signature-based sensor library (database of signatures) must be updated constantly to keep up with the evolving variety and nature of internet attacks. An updated sensor will generally detect any attacks present in the network traffic for which it contains a valid signature. Additionally, false positive rates for signature-based sensors are usually low, meaning that they do not normally alert on normal (non-attack) network traffic.
One limitation of this method of attack detection is that it is based on a retrospective view; signature-based sensors can only detect known attacks for which they have an accurate signature in their libraries. All signature-based sensors exhibit blind spots which are defined by either categories of attacks for which no signatures have been established, or “zero day exploits” (new attacks) which have never been seen before. Additionally, all signature-based sensors tend to have similar blind spots because they typically subscribe to the same reference libraries of known malicious behavior. Two of the most widely used subscription libraries can be found at Snort.org and Symantec.com.
The blind spot problem for signature-based sensors is compounded by the fact that use of evasion techniques by hackers has proven very effective at enabling known exploits to escape detection. Evasion techniques allow a hacker to sufficiently modify the pattern of an attack so that the signature will fail to produce a match (during intrusion detection). The most common evasion techniques are obfuscation, fragmentation, and encryption. Obfuscation is hiding intended meaning in communication, making communication confusing, willfully ambiguous, and harder to interpret. In network security, obfuscation refers to methods used to obscure an attack payload from inspection by network protection systems. For instance, an attack payload can be hidden in web protocol traffic. Fragmentation is breaking a data stream into segments and sending those segments out of order through the computer network. The segments are reassembled in correct order at the receiving side. The shuffling of the order of data stream segments can change the known attack signature due to the reordering of communication bits. Encryption is the process of encoding messages (or information) in such a way that eavesdroppers or hackers cannot read it, but that authorized parties can. Both the authorized sender and receiver must have the same encryption key because the process of encoding and decoding is relatively the same. In network attacks, the attack payload can often be encoded/encrypted such that the signature is no longer readable by detection systems. While each evasion technique changes the attack pattern differently, it is important to note that the goal is the same: change the attack pattern enough to no longer match published attack signatures and hence to avoid intrusion detection.
Another type of intrusion detection system (sensor), which does not rely on signature libraries, is starting to be deployed on computer networks. These non-signature-based sensors use mathematical algorithms to perform either anomaly detection or categorical classification (machine learning). In anomaly detection, the sensor reviews network traffic, uses algorithms (mathematical methods) to form a mathematical model to represent the network traffic, and then alerts on any traffic that does not fit the model (anomalous). Anomaly detectors also use thresholds and rule sets to help bound the normal network traffic space, that is, to determine whether network traffic is normal or not. Machine-learning sensors are similar in that they use an algorithm to form a model, but they separate network traffic into multiple categories for each variant of network traffic (as opposed to an anomaly detector which uses only one category: “normal”). The model is automatically created by using the algorithm to process previously-tagged training samples containing network traffic from multiple categories. After such training, the machine learning sensor will review and analyze network traffic by placing each portion of network traffic into its “best fitting” category.
In both anomaly detection and categorical classification (machine learning), models are preferably created automatically using well known techniques, such as those disclosed in Bremner D, Demaine E, Erickson J, Iacono J, Langerman S, Morin P, Toussaint G (2005), “Output-sensitive algorithms for computing nearest-neighbor decision boundaries”, Discrete and Computational Geometry 33 (4): 593-604. doi:10.1007/s00454-004-1152-0, and D. Coomans; D. L. Massart (1982). “Alternative k-nearest neighbour rules in supervised pattern recognition: Part 1. k-Nearest neighbour classification by using alternative voting rules”. Analytica Chimica Acta 136: 15-27. doi:10.1016/S0003-2670(01)95359-0, both of which are hereby incorporated herein by reference.
These algorithm-based (anomaly detection and machine-learning) methods of sensing do not depend on referencing man-made signatures, but instead depend on the ability of the algorithms to learn about and recognize both normal and malicious traffic. During the learning process, the anomaly detection and machine-learning sensors are exposed to combinations of known normal and malicious traffic samples, so that their mathematical algorithms can develop a mathematical model of the network traffic. This process is known as sensor training. The benefit of using algorithm based (anomaly detection and machine-learning) sensors is that they can alert on either known attacks (even when evasion techniques are employed by the attacker) or on new attacks/exploits (zero-days) that have never been used before.
However promising the use of algorithm-based sensors may be, in reality most algorithm-based sensors experience high false-positives rates. The more complex the computer network, the more difficult it is to train the algorithms to achieve acceptable performance levels to support effective computer network defense. An example of a high false-positive problem would be when a sensor alert report shows 10 attacks occurring, when there are only 5 actual attacks. This would be a false-positive rate of 50%. In a network defensive operation, where analysts must investigate every alert, time and effort is wasted investigating false alerts (false-positives). Thus a sensor with a high false-positive rate is not useful to network defense operations.
Computer network defense of large enterprise networks is a persistent problem that is growing more difficult and important to address as the prosperity of modern society becomes increasingly more dependent on networked computer systems. Network defense analysts know they are not catching all attacks and recognize that the problem is getting worse. Once a new attack is identified, defense personnel have to wait until subscription publishers update signature libraries before any protection is available. Typically, new attacks are not caught immediately and there is often a significant delay of from days to weeks before signatures are available for distribution to defense personnel. This retrospective view is inadequate for effective network intrusion defense.
A system is needed to provide the ability to detect the full spectrum of network attacks: known, pattern obscured, and “zero-day” attacks, with an acceptable false-positive rate. Algorithm-based sensors, combined with signature-based sensing, can provide this full spectrum of network defense while eliminating the blind spot and false positive rate problems each separate sensor experiences. We define this heterogeneous sensor combination as operationally using two sensors of different underlying inspection methods to detect the same threat. Also, for simplicity, the term “machine learning” shall hereinafter mean and include both anomaly detection and machine learning sensors.